Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic.

Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.

Patient Interaction Phenotypes With an Automated Remote Hypertension Monitoring Program and Their Association With Blood Pressure Control: Observational Study.

BACKGROUND: Automated texting platforms have emerged as a tool to facilitate communication between patients and health care providers with variable effects on achieving target blood pressure (BP). Understanding differences in the way patients interact with these communication platforms can inform their use and design for hypertension management. OBJECTIVE: Our primary aim was to explore the unique phenotypes of patient interactions with an automated text messaging platform for BP monitoring. Our secondary aim was to estimate associations between interaction phenotypes and BP control. METHODS: This study was a secondary analysis of data from a randomized controlled trial for adults with poorly controlled hypertension. A total of 201 patients with established primary care were assigned to the automated texting platform; messages exchanged throughout the 4-month program were analyzed. We used the k-means clustering algorithm to characterize two different interaction phenotypes: program conformity and engagement style. First, we identified unique clusters signifying differences in program conformity based on the frequency over time of error alerts, which were generated to patients when they deviated from the requested text message format (eg, ###/## for BP). Second, we explored overall engagement styles, defined by error alerts and responsiveness to text prompts, unprompted messages, and word count averages. Finally, we applied the chi-square test to identify associations between each interaction phenotype and achieving the target BP. RESULTS: We observed 3 categories of program conformity based on their frequency of error alerts: those who immediately and consistently submitted texts without system errors (perfect users, 51/201), those who did so after an initial learning period (adaptive users, 66/201), and those who consistently submitted messages generating errors to the platform (nonadaptive users, 38/201). Next, we observed 3 categories of engagement style: the enthusiast, who tended to submit unprompted messages with high word counts (17/155); the student, who inconsistently engaged (35/155); and the minimalist, who engaged only when prompted (103/155). Of all 6 phenotypes, we observed a statistically significant association between patients demonstrating the minimalist communication style (high adherence, few unprompted messages, limited information sharing) and achieving target BP (P<.001). CONCLUSIONS: We identified unique interaction phenotypes among patients engaging with an automated text message platform for remote BP monitoring. Only the minimalist communication style was associated with achieving target BP. Identifying and understanding interaction phenotypes may be useful for tailoring future automated texting interactions and designing future interventions to achieve better BP control.

Sparse multi-output Gaussian processes for online medical time series prediction

BACKGROUND: For real-time monitoring of hospital patients, high-quality inference of patients'health status using all information available from clinical covariates and lab test results is essential to enable successful medical interventions and improve patient outcomes. Developing a computational framework that can learn from observational large-scale electronic health records (EHRs) and make accurate real-time predictions is a critical step. In this work, we develop and explore a Bayesian nonparametric model based on multi-output Gaussian process (GP) regression for hospital patient monitoring. METHODS: We propose MedGP, a statistical framework that incorporates 24 clinical covariates and supports a rich reference data set from which relationships between observed covariates may be inferred and exploited for high-quality inference of patient state over time. To do this, we develop a highly structured sparse GP kernel to enable tractable computation over tens of thousands of time points while estimating correlations among clinical covariates, patients, and periodicity in patient observations. MedGP has a number of benefits over current methods, including (i) not requiring an alignment of the time series data, (ii) quantifying confidence regions in the predictions, (iii) exploiting a vast and rich database of patients, and (iv) inferring interpretable relationships among clinical covariates. RESULTS: We evaluate and compare results from MedGP on the task of online prediction for three patient subgroups from two medical data sets across 8,043 patients. We find MedGP improves online prediction over baseline and state-of-the-art methods for nearly all covariates across different disease subgroups and hospitals. CONCLUSIONS: The MedGP framework is robust and efficient in estimating the temporal dependencies from sparse and irregularly sampled medical time series data for online prediction. The publicly available code is at https://github.com/bee-hive/MedGP .

Locally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic challenges hospital leaders to make time-sensitive, critical decisions about clinical operations and resource allocations. OBJECTIVE: To estimate the timing of surges in clinical demand and the best- and worst-case scenarios of local COVID-19-induced strain on hospital capacity, and thus inform clinical operations and staffing demands and identify when hospital capacity would be saturated. DESIGN: Monte Carlo simulation instantiation of a susceptible, infected, removed (SIR) model with a 1-day cycle. SETTING: 3 hospitals in an academic health system. PATIENTS: All people living in the greater Philadelphia region. MEASUREMENTS: The COVID-19 Hospital Impact Model (CHIME) (http://penn-chime.phl.io) SIR model was used to estimate the time from 23 March 2020 until hospital capacity would probably be exceeded, and the intensity of the surge, including for intensive care unit (ICU) beds and ventilators. RESULTS: Using patients with COVID-19 alone, CHIME estimated that it would be 31 to 53 days before demand exceeds existing hospital capacity. In best- and worst-case scenarios of surges in the number of patients with COVID-19, the needed total capacity for hospital beds would reach 3131 to 12 650 across the 3 hospitals, including 338 to 1608 ICU beds and 118 to 599 ventilators. LIMITATIONS: Model parameters were taken directly or derived from published data across heterogeneous populations and practice environments and from the health system's historical data. CHIME does not incorporate more transition states to model infection severity, social networks to model transmission dynamics, or geographic information to account for spatial patterns of human interaction. CONCLUSION: Publicly available and designed for hospital operations leaders, this modeling tool can inform preparations for capacity strain during the early days of a pandemic. PRIMARY FUNDING SOURCE: University of Pennsylvania Health System and the Palliative and Advanced Illness Research Center.